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IMPLEMENTATION  OF  A  THEORY  FOR  INFERRING 
SURFACE  SHAPE  FROM  CONTOURS 

Kent  A.  Stevens1 


ABS  TRACT.  Human  vision  is  adept  at  inferring  the  shape  of  a  surface  from  the  image  of  curves  lying  across 
the  surface.  The  strongest  impression  of  3-1)  shape  derives  from  parallel  (but  not  necessarily  equally  spaced) 
contours.  In  [StcVcns  198  la]  the  computational  problem  of  inferring  3-1)  shape  from  image  configurations  is 
examined,  and  a  theory  is  given  for  how  the  visual  system  constrains  the  problem  by  certain  assumptions. 
The  assumptions  arc  three:  that  neither  the  viewpoint  nor  the  placement  of  the  physical  curves  on  the  surface 
is  misleading,  and  that  the  physical  curves  are  lines  of  curvature  across  the  surface.  These  assumptions  imply 
that  parallel  image  contours  correspond  to  parallel  curves  lying  across  an  approximately  cylindrical  surface. 
Moreover,  lines  of  curvature  on  a  cylinder  are  geodesic  and  planar.  These  properties  provide  strong 
constraint  on  the  local  surface  orientation.  We  describe  a  computational  method  embodying  these  geometric 
constraints  that  is  able  to  determine  the  surface  orientation  even  in  places  where  locally  it  is  very  weakly 
constrained,  by  extrapolating  from  places  where  it  is  strongly  constrained.  This  computation  has  been 
implemented,  and  predicts  local  surface  orientation  that  closely  matches  the  apparent  orientation. 
Experiments  with  the  implementation  support  the  theory  that  our  visual  interpretation  of  surface  shape  from 
contour  assumes  the  image  contours  correspond  to  lines  of  curvature. 
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1.  Introduction 

Hie  human  visual  system  is  adept  at  inferring  the  shape  of  a  surface  from  an  image  consisting  only  of 
contours  presumed  to  correspond  to  physical  curves  lying  on  the  surface.  In  figure  1,  for  example,  we 
perceive  a  rippled  surface.  The  contours  that  comprise  this  figure  are  interpreted  as  arrayed  in  3-D  on  a 
smooth  but  otherwise  invisible  surface. 

Several  questions  arc  posed  by  figure  1.  Most  immediately,  why  do  we  take  a  3-1)  interpretation  instead  of 
the  (literal)  interpretation  of  figure  1  as  sinusoids  on  the  plane  of  the  printed  page?  The  2-D  interpretation  is 
almost  impossible  to  hold,  the  tendency  to  see  depth  in  the  figure  is  so  strong.  Moreover,  our  3-D 
interpretation  is  of  a  particular  surface  shape.  Kxcept  for  the  expected  depth  reversal  in  figure  1  (which 
changes  the  apparent  orientation  of  the  surface),  different  observers  see  much  the  same  rippled  surface.  What 
then  determines  the  specific  3-D  interpretation  that  we  take?  Clearly  we  incorporate  geometric  constraints 
strong  enough  to  force  a  particular  3-D  shape.  What  arc  they? 

ITiis  article  starts  with  a  review  of  a  theory  [Stevens  1981a]  regarding  the  specific  geometric  constraints 
underlying  the  interpretation  of  surface  contours.  As  we  shall  see,  a  few  constraints  arc  sufficient  to  allow  the 
determination  of  local  surface  shape  from  parallel2  contours  and  certain  related  image  configurations.  Can  we 
find  evidence  that  our  visual  interpretation  incorporates  these  particular  geometric  constraints?  ITic  intent  of 
this  article  is  largely  empirical,  and  centers  on  experiments  that  compare  human  performance  with  the  3-D 
shape  computed  by  an  implementation  of  the  theory.  The  observed  similarity  supports  the  theory. 

2.  Review  of  a  theory  of  constraints  on  surface  contour 

2.1  Surface  contours  and  their  physical  counterparts 

A  given  contour  in  a  natural  image  corresponds  to  perhaps  an  object  edge,  a  crease,  a  shadow,  or  some  surface 


figure  1  A  sci  of  parallel  contours  that  are  interpreted  in  3-D  as  lying  on  a  smooth  cylindrical  surface  The  surface  shape  is  vivid  and 
one  can  judge  with  confidence  the  local  surface  orientation.  This  suggests  that  specific  gcomciric  constraints  are  imposed  on  Ihc  3-D 
interpretation.  Assuming  that  the  coniours  arc  lines  of .  ealest  curvature  on  (he  surface  plus  weaker  ancillary  assumptions  provides 
sufficient  constraint  on  the  problem  of  inferring  local  surface  orientation  from  such  configurations. 


2  Recall  that  two  arbitrary  curves  arc  parallel  if  one  can  be  superimposed  upon  the  other  simply  by  a  translation. 
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pigmentation  marking.  Kadi  image  contour  has  a  specific  physical  cause.  In  this  treatment,  however,  we  will 
consider  a  surface  contour  only  geometrically,  as  a  curve  in  2-1)  that  corresponds  to  some  curve  in  3-1), 
without  regard  for  what  the  physical  curve  may  be.  Of  course,  knowledge  of  the  physical  curve  would  be 
expected  to  simplify  the  interpretation  task,  but  human  vision  does  not  rely  on  knowing  this  information:  we 
infer  a  definite  3-1)  shape  from  purely  abstract  configurations  of  curves,  as  figure  1  demonstrates.  (In  the 
absence  of  information  about  the  corresponding  physical  curves,  we  might  take  some  default  assumptions 
about  diem,  but  it  is  difficult  to  surmise  a  priori  what  those  assumptions  are.) 

Not  only  do  we  infer  surface  shape  from  abstract  lines,  but  the  lines  need  not  even  be  continuous  in  the 
image.  In  figure  2b  the  contours  arc  defined  by  dots,  and  in  figure  2c  die  contours  arc  "subjective  contours" 
defined  by  line  terminations.  The  three  surfaces  seen  in  figure  2  arc  very  similar.  Thus  we  conclude  diat 
surface  shape  is  inferred  from  a  radicr  general  representation  of  contour. 

2.2  Terminology 

The  subsequent  discussions  largely  concern  the  geometric  properties  of  a  particular  type  of  contour  (the  line 
of  curvature)  lying  across  a  particular  class  of  surface  (the  cylinder).  A  brief  discussion  of  these  concepts  is 
given  below  (sec  also  e.g.  [Hilbert  &  Cohn-Vosseti  1952]). 

Consider  a  smooth  patch  of  surface  and  a  plane  that  intersects  the  surface  perpendicularly.  That  is.  the 
surface  normal  lies  in  die  plane.  The  intersection  of  the  surface  and  the  plane  defines  a  curve  whose 
curvature  is  called  normal  curvature.  Suppose  die  plane  is  rotated,  with  the  surface  normal  the  axis  of 
rotation.  Depending  on  the  surface  in  question,  the  normal  curvature  at  the  given  point  will  vary  between  a 
minimum  and  maximum,  called  the  principal  curvatures.  The  tangent  to  die  curve  when  die  curvature  is 
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cither  maximum  or  minimum  will  point  in  one  or  the  other  of  two  principal  directions,  and  for  points  on 
smooth  surfaces  the  principal  directions  arc  mutually  perpendicular.  Thus  each  point  on  a  smooth  surface  has 
two  distinguished  directions  (except  where  the  surface  patch  is  planar  or  spherical,  in  which  case  the  normal 
curvature  is  constant  in  all  directions:  such  points  arc  called  umbilic). 

'Hie  lines  of  curvature  arc  curves  across  a  given  surface  that  follow  one  or  the  other  principal  directions. 
Specifically,  the  tangent  at  each  point  along  a  line  of  greatest  (or  least)  curvature  aligns  with  the  direction  of 
greatest  (or  least)  normal  curvature.  Since  the  two  principal  directions,  where  defined,  arc  mutually 
perpendicular,  the  lines  of  greatest  and  least  curvature  form  an  orthogonal  net 

Of  particular  interest  to  us  will  be  surfaces  for  which  the  lines  of  least  curvature  arc  straight  lines,  the 
so-called  singly-curved  or  developable  surfaces.  Such  surfaces  correspond  to  the  configurations  that  one  can 
make  with  a  sheet  of  paper,  allowing  bending  and  twisting  but  no  tearing  or  creasing.  On  a  singly-curved 
surface  the  tangent  to  a  line  of  greatest  curvature  points  in  lire  direction  in  which  the  surface  bends  most 
rapidly,  and  for  a  line  of  least  curvature  die  tangent  points  in  llic  direction  of  zero  normal  curvature. 

A  cylinder  is  a  restriction  on  the  singly-curved  surfaces  wherein  twisting  is  disallowed.  That  means  the 
lines  of  least  curvature  (also  called  rulings )  arc  all  parallel.  The  cylinder  corresponds  to  a  rolled  or  rippled 
sheet  of  paper  or  a  hanging  curtain.  The  lines  of  greatest  curvature,  lying  perpendicular  to  the  direction  of  the 
ripples,  arc  sometimes  called  cross  sections. 

The  lines  of  curvature  on  a  cylinder  have  two  important  properties:  they  arc  planar  and  geodesic.  Being 
geodesic,  die  principal  normal  to  the  line  of  greatest  curvature  is  identically  the  surface  normal  at  that  point. 
And  since  the  curve  is  planar,  die  surface  normal  along  diat  path  is  restricted  to  rotating  in  the  plane 
containing  the  curve  (the  osculating  plane). 

2.3  Surface  contours  arc  distinct  from  texture  contours  and  carry  different  >1)  information 
An  important  distinction  exists  between  the  contours  we  will  examine  here  and  diosc  diat  comprise  image 
texture.  While  both  correspond  to  physical  curves  across  surfaces  in  3-D  and  both  contribute  surface  shape 
information,  dicy  differ  in  the  way  they  "encode"  that  information. 

Surface  texture  is  foreshortened  when  the  surface  is  slanted  relative  to  the  line  of  sight.  The  amount  and 
direction  of  the  foreshortening  carries  local  information  about  the  amount  and  direction  of  the  surface  slant 
Thus  local  surface  orientation  can  be  estimated  from  texture  foreshortening,  provided  die  foreshortening  can 
be  quantified  in  die  image,  and  provided  certain  physical  restrictions  in  the  physical  texture  arc  met.  Harly 
proposals  for  quantifying  texture  foreshortening  (reviewed  in  [Mock  1964])  were  in  terms  of  the 
height-to-width  ratio,  which  varies  with  the  cosine  of  the  slant  angle  in  the  ideal  case  of  circular  and 
flush-lying  surface  features.  Recently,  Witkin  [1981]  has  shown  diat  surface  orientation  can  be  estimated  from 
local  statistics  of  texture  contour.  A  representative  example  of  where  contour  foreshortening  carries 
information  about  surface  shape  is  given  by  the  mottled  pattern  of  sunlight  and  shadows  cast  on  the  ground 
below  a  tree.  Instead  of  examining  the  height-to-width  ratio  of  the  individual  patterns  of  light  and  dark,  the 
method  examines  the  contours  that  follow  die  boundaries  between  light  and  dark.  ITte  contour  curvature 
varies  in  a  statistical  but  systematic  manner  with  tangent  direction  and  surface  orientation  [Witkin  1981]. 
Hence  from  measurements  of  these  statistics  the  local  surface  orientation  can  be  estimated.  The  accuracy  of 
the  estimate  depends  on  several  factors.  First,  the  physical  texture  must  lie  flush  (as  do  molded  shadows  on 
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tlic  smooth  ground)  so  tJiat  they  foreshorten  according  to  slant  angle  in  the  expected  manner  (see  (Stevens 
1981b]  for  further  discussion).  Second,  the  curvature  statistics  of  the  physical  texture  must  be  isotropic, 
otherwise  tlic  texture  anisotropy  would  resemble  foreshortening  and  misleadingly  appear  slanted,  lliinl. 
since  tlic  method  is  statistical,  the  surface  must  be  roughly  planar  in  each  region  over  which  the  statistics  arc 
gathered,  otherwise  surface  curvature  would  confound  tlic  foreshortening  interpretation.  Witkin  (1981] 
demonstrates  this  computational  method  on  slanted  planar  outlines  and  observes  that  if  applied  along  the 
contour  of  an  ellipse,  the  method  estimates  the  spatial  orientation  of  the  corresponding  circular  disk. 

Some  method  for  inferring  surface  orientation  from  contour  foreshortening  similar  to  (Witkin  1981]  is 
likely  incorporated  in  human  vision,  and  is  applied  to  image  configurations  in  which  the  contours  appear 
foreshortened  on  a  planar  surface,  provided  there  is  not  belter  evidence  to  tlic  contrary,  f  igure  3  is  an 
example:  die  pattern  resembles  a  sheet  of  plywood  or  waves  lapping  at  a  beach.  That  is  not  the  only  3-D 
interpretation  available  in  this  pattern,  however.  Like  figure  1.  figure  3  may  also  be  seen  as  an  undulating 
surface,  but  because  of  the  amplitude  of  the  curves,  the  surface  dips  dramatically  from  crest  to  trough. 

Figure  3  is  rotated  by  90°.  relative  to  figure  1.  to  bias  its  interpretation  as  a  planar  surface  over  diat  of  an 
undulating  surface.''  While  we  strongly  prefer  the  planar  interpretation  in  figure  3.  one  can  also  interpret  it  as 
a  deeply  convoluted  surface  (as  noted,  view  it  so  dial  die  peaks  and  troughs  would  be  oriented  as  in  figure  1). 
As  the  amplitude  of  die  sinsuoid  pattern  increases  the  planar-surface  interpretation  tends  to  dominate  over 
the  undulating-surface  interpretation,  however  there  is  a  rather  broad  intermediate  range  for  which  the  two 
interpretations  arc  strongly  rivalrous.  In  this  range  the  interpretation  taken  depends  strongly  on  die 
orientation  of  the  figure. 

The  two  cases  have  a  neat  geometric  interpretation  in  terms  of  the  physical  curves.  In  the  planar-surface 
case  the  physical  curves  arc  essentially  asymptotic  lines;  in  the  undulating-surface  case  they  arc  geodesic  (as 
will  be  discussed).  Asymptotic  lines  have  zero  normal  curvature,  all  of  their  curvature  lies  in  the  tangent 
plane.  In  contrast,  geodesic  lines  have  zero  curvature  ir  "  tangent  plane,  all  of  dicir  curvature  is  normal 
curvature.  Thus  die  two  interpretations  will  be  appreciate  as  b  ng  die  two  extremes  of  a  continuum.  We 
will  use  the  terms  "texture  contours"  and  "surface  contours"  to  disdnguish  die  two  cases. 

2.4  Surface  shape  is  described  by  local  surface  orientation 

It  is  proposed  that  surface  orientation  is  the  primary  form  of  3-D  shape  information  derived  from  contours 
such  as  diosc  in  figure  1.  Several  factors  support  diis  proposal:  it  is  feasibly  computed,  sufficient  for  deriving 
more  global  shape  descriptions,  and  the  apparent  surface  orientadon  is  closely  predicted  by  the  theory  of 

3  It  is  well  known  that  in  most  natural  scenes  distance  increases  as  one  scans  from  the  bottom  to  the  top  of  the  visual  Held,  a  simple 
consequence  of  our  conventional  viewpoint  of  objects  on  the  horizontal  ground.  In  terms  of  surface  orientation,  this  tendency  is  reflected 
in  a  bias  to  choosing  an  upward  pointing  surface  normal.  Lor  illustration,  consider  viewing  an  ellipse  with  horizontal  major  axis:  if 
interpreted  as  a  ciicular  disk  on  the  ground  the  normal  would  point  vertically  upward  Hut  if  the  ellipse's  major  axis  is  vertical,  so  that 
the  noi  trial  would  point  to  either  the  left  or  right.  I  have  observed  no  apparent  bias  to  one  interpretation  over  Ore  other  Ihc  result  is  that 
when  the  apparent  surface  normal  points  vertically  the  depth  reversals  are  less  frequent  than  when  it  points  horizontally.  Ibis  biases  can 
be  demonstiatcd  for  a  wide  range  of  stimulus  surfaces,  and  is  used  lo  advantage  in  figure  3  Ihe  practical  consequence  is  that  the 

O 

planar-surface  interpretation  in  figure  3  is  strongest  when  the  apparent  surface  is  horizontal.  Observe  that  by  rotating  figure  3  by  00  the 
undulating-surface  interpolation  is  easier  to  achieve  and  lire  planar-surface  interpretation,  when  achieved,  is  less  stable  because  of  depth 
reversals.  Similarly,  rotating  figure  I  so  that  the  normals  point  roughly  horizontally,  rather  than  vertically,  results  in  more  licqueni  depth 
reversals. 
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Pigurc  3.  The  curves  tend  to  be  interpreted  as  lying  on  a  planar  surface,  although  they  may  also  be  seen  to  lie  across  a  dramatically 
undulating  surface.  I  he  relative  strength  of  the  two  spatial  interpretations  varies  with  the  orientation  of  the  figure  (see  text). 


inferring  surface  orientation  from  contours.  The  computational  feasibility  will  be  reviewed  momentarily; 
simply  suited,  local  surface  orientation  is  rather  immediately  encoded  in  the  image  while  distance  information 
is  not.  Surface  orientation  also  provides  a  sufficient  basis  from  which  other  surface  shape  descriptions  can  be 
subsequently  derived  such  as  disuincc  (up  to  an  additive  constant  and  an  overall  multiplicative  scalar),  surface 
curvature,  and  topographical  features  such  as  ridges  and  troughs.  The  close  match  between  apparent  surface 
orientation  and  that  predicted  by  the  theory  will  be  demonstrated  later. 

Surface  orientation,  having  two  degrees  of  freedom,  can  be  regarded  as  a  vector  quantity  having  magnitude 
and  direction.  The  magpitude  component  is  the  familiar  variable  slant,  the  angle  a  between  the  line  of  sight 
and  the  local  surface  normal.  Slant  varies  over  a  range  from  0°  (where  the  surface  patch  is  perpendicular  to 
the  line  of  sight)  to  90°.  The  other  degree  of  freedom  concerns  the  direction  of  slanu  called  tilt  (Stevens 
1979).  The  tilt  t  is  the  direction  in  which  the  surface  normal  would  project  in  the  image,  and  also  corresponds 
to  the  direction  of  the  gradient  of  distance  from  the  viewer  to  the  surface.  Slant  and  tilt  will  be  used  to 
quantify  orientation  for  our  purposes. 

The  apparent  surfaces  suggested  by  contours  in  these  figures  arc  subject  to  the  familiar  ’’depth"  reversals 
usually  associated  with  die  Neckcr  cube.  These  reversals  arc  expected,  and  correspond  straightforwardly  to 
ambiguity  in  the  tilt  direction.  That  is,  the  orientation  of  the  surface  can  be  recovered  only  to  within  a 
reflection  in  depth  about  the  image  plane,  or  equivalently,  to  within  a  180°  reversal  in  the  direction  the 
surface  normal  would  point  locally.  Hach  choice  of  direction  indicates  the  direction  of  the  gradient  of 
distance.  While  the  subjective  impressions  of  depth  and  of  surface  shape  may  change  dramatically  with  these 
reversals,  it  amounts  to  only  a  reversal  in  the  tilt  component,  the  slant  angle  is  unchanged.  (It  is  worthwhile 
examining  the  tilt  ambiguity  in  the  various  illustrations  of  this  article,  in  order  to  convince  oneself  that  when 
the  apparent  depth  reverses,  the  till  component  of  the  surface  orientation  for  any  given  patch  reverses 
direction.)  In  light  of  this  ambiguity,  we  expect  to  be  able  to  compute  surface  tilt  only  to  within  a  180° 
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reversal. 

2.5  Hie  computational  problem 

Hie  perceptual  task  we  accomplish  so  effortlessly  when  viewing  these  figures  can  be  described  as  a 
computational  problem  (in  the  sense  of  [Marr  &  Poggio  1977;  Marr  1982])  as  follows.  We  are  presented  with 
a  set  of  contours  {C}  in  an  image,  assumed  to  correspond  to  a  set  of  physical  curves  { T}  across  a  surface  2. 
I  he  task  is  to  determine  the  local  surface  orientation  of  the  visible  patches  of  2,  where  surface  orientation 
relative  to  the  viewer  is  quantitifed  by  slant  and  tilt. 

In  studying  how  we  accomplish  this  task,  our  approach  will  be  to  concentrate  first  on  the  ease  where  the 
contours  are  parallel  in  the  image,  as  in  figure  1,  then  show  how  this  problem  extends  naturally  to  certain 
other  configurations. 

Our  tendency  to  take  one  specific  3-D  interpretation  in  viewing  figure  1  is  so  strong,  we  must  be  reminded 
that  the  problem  is  highly  underconstrained.  Hicrc  are  infinitely  many  3-1)  surfaces  that  consistent  with  the 
given  2-1)  projection,  including  die  surface  of  die  page  on  which  it  is  printed.  What  constrains  die  particular 
interpretation  diat  we  take?  We  have  no  expectations  of  viewing  any  specific  surface  shape,  and  no 
expectations  for  viewpoint  relative  to  the  surface.  Hie  central  theoretical  issue  that  emerges,  then,  is 
discovering  what  constraints  arc  incorporated  in  the  method  adopted  by  the  human  visual  system  in  "solving" 
this  problem  of  3-D  perception. 

Observe  that  figure  l  evokes  the  impression  of  viewing  a  continuous  but  transparent,  or  invisible,  surface. 
One  has  a  strong  sense  for  die  surface  shape  in  the  spaces  between  the  contours  as  well  as  along  them.  We 
will  be  concerned  primarily  with  determining  orientation  only  for  those  surface  points  along  a  given  physical 
curve  T.  In  die  intervening  regions  (where  there  is  no  surface  information  available)  die  orientation  is 
presumably  determined  by  interpolation.  In  the  ease  of  cylindrical  surfaces,  which  arc  important  here,  the 
interpolation  will  be  seen  to  be  trivial.  Hie  more  general  ease  of  interpolating  doubly-curved  surfaces  is 
beyond  die  scope  of  this  article. 

2.6  Constraining  the  3-1)  interpretation 

In  [Stevens  1981a[  it  is  proposed  that  our  interpretation  is  constrained  primarily  by  assuming  diat  die  contours 
correspond  to  lines  of  curvature  on  the  surface.  That  strong  assumption,  plus  two  assumptions  to  the  effect 
'.hat  neither  the  viewpoint  nor  the  particular  placement  of  die  physical  curves  on  die  surface  is  misleading, 
leads  to  an  effective  method  for  determining  surface  orientation  from  certain  configurations. 

2.6.1  General  position  of  viewpoint:  Ihe  surface  2  and  the  physical  curves  across  2  are  assumed  to  be  seen 
from  a  representative  viewpoint,  i.c.  the  image  projection  diat  results  from  this  viewpoint  is  similar  to  the 
which  would  result  from  a  slightly  different  viewpoint.  By  assuming  this,  we  may  exclude  die  following 
degenerate  eases  of  projection:  i)  a  planar  curve  projecting  as  a  straight  line,  ii)  two  non-collinear  lines 
projecting  as  collincar,  and  iii)  two  non-parallel  curves  projecting  as  parallel.  ITius  if  a  curve  is  straight  in  the 
2-D  image  it  is  straight  in  3-D,  and  likewise  for  two  collincar  or  parallel  lines  in  2-1)  and  their  3-D 
counterparts. 

Ihe  a  priori  probability  of  encountering  any  of  the  above  three  degeneracies  is  small.  It  is  particularly 
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difficult  for  multiple,  non-parallel  curves  appear  parallel  in  the  image.  As  the  complexity  of  the  projected 
curves  increases  it  becomes  increasingly  improbable  that  two  different  and  thus  non-parallel  curves  would  be 
placed  fortuitously  relative  to  each  other  and  relative  to  the  viewer  so  that  their  projections  would  differ  only 
by  a  translation  in  the  image.  lhc  other  two  degenerate  eases,  we  recall,  arc:  i)  a  str  aight  line  in  the  image  that 
is  actually  die  projection  of  a  planar  curve,  and  ii)  tw  o  collinear  lines  in  die  image  that  are  die  projection  of 
two  lines  dint  lie  in  die  same  plane  but  are  non-collincar.  Both  are  coincidental  for  the  same  reason:  the  line 
of  sight  happens  to  lie  in  die  plane  containing  die  planar  curve  or  the  two  lines. 

2.6.2  Genera 1  position  of  contour:  lhc  surface  geometry  in  the  vicinity  of  the  physical  curve  is  assumed  to  be 
similar  to  that  directly  under  the  curve.  Consider  figure  1,  where  the  surface  between  contours  is  invisible. 
An  interpretation  that  we  do  not  take  is  diat  die  surface  varies  substantially  between  the  contours.  Instead, 
our  interpretation  is  diat  the  particular  placement  of  die  curves  on  the  surface  is  not  critical  or  fortuitous,  so 
that  if  dicy  were  shifted  laterally  die  image  would  appear  substantially  the  same. 

lhis  assumption,  plus  the  earlier  assumption  that  die  viewpoint  is  representative,  allows  one  to  conclude 
that  in  the  v  icinity  of  parallel  physical  contours  die  underlying  surface  patches  arc  approximately  cylinderical 
(see  below).  General  position  of  contour  is  related  to  Grinison’s  [1981]  observation  that  significant  changes  in 
surface  geometry  arc  usually  reflected  by  intensity  changes  in  the  image.  That  is,  if  die  surface  deviates 
significantly  from  cylindricalily  between  a  given  pair  of  parallel  contours,  it  would  usually  be  apparent  in  die 
image.  Thus  in  die  absence  of  evidence  to  the  contrary,  cylindricality  can  be  assumed.  This  assumption 
seems  a  priori  reasonable  in  the  restricted  context  of  parallel  curves:  in  most  instances  w  here  parallel  curves 
lie  on  a  surface,  the  surface  in  the  vicinity  is  cylindrical. 

2.6.3  The  line  of  cunaturc  assumption:  The  physical  curve  T  is  assumed  to  be  a  line  of  curvature  across  2,  Lc., 
the  tangent  to  T  everywhere  coincides  with  one  of  the  principal  directions  on  the  surface.  In  die  ease  of 
parallel  contours,  since  die  surface  in  die  vicinity  is  a  cylinder,  it  follows  immediately  dial  V  is  also  planar  and 
geodesic. 

lhc  a  priori  justification  of  assuming  surface  contours  arc  lines  of  curvature  is  rather  difficult.  But  as 
observed  in  [Stevens  1981a[.  it  is  interesting  that  die  vast  majority  of  curves  across  the  surfaces  of  synthetic 
objects  of  all  sorts  arc  lines  of  curvature.  The  reasons  for  this  are  many,  and  include  the  following:  most 
manufactured  objects  arc  composed  of  surfaces  of  revolution  or  cylinders,  if  diey  arc  not  planar,  and  the 
markings  placed  on  these  surfaces  of  revolution  and  cylinders  almost  invariably  follow  die  cross  sections, 
meridians,  and  rulings  --  all  of  which  arc  intrinsically  lines  of  curvature.  Not  only  arc  markings  across  the 
surface  usually  lines  of  curvature,  but  the  seams,  creases,  and  edges  are  as  well.  Even  in  eases  where  die 
surface  geometry  is  more  complex,  such  as  metal  castings  and  plastic  injection-moldings,  the  scams  arc  usually 
planar,  normal  sections,  and  lines  of  curvature  (although  this  is  sometimes  not  as  immediately  obvious  by 
inspection  as  are  the  meridians  on  a  surface  of  rcvoludon  or  the  cross  sections  of  a  cylinder). 

In  the  same  vein,  the  contours  diat  arise  in  biological  forms  arc  very  often  lines  of  curvature.  The  joints  in 
a  bamboo  stalk  are  cross  sections  and  the  markings  that  run  lengthwise  along  the  stalk  are  rulings  --  both  of 
which  are  lines  of  curv  ature.  Stripes  and  linear  markings  on  vegetation  for  various  reasons,  are  often  aligned 
with  die  principal  directions  on  the  surface.  In  fact,  wrinkles  in  skin  accompanied  with  flexure  at  joints  or 
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compression  due  to  underlying  muscles  tend  to  follow  lines  of  curvature.  A  final  observation  should  suffice: 
on  specular  surfaces,  either  inherently  glossy  or  merely  wet.  the  highlights  either  appear  pointlike  or 
extended.  In  the  case  of  extended  specular  reflections,  if  they  appear  as  straight  in  the  image,  the  surface  is 
locally  a  cylinder,  and  the  path  on  the  surface  corresponding  to  the  specularity  is  a  line  of  least  curvature  (see 
[Stevens  1981a]). 

The  difficulty  of  this  sort  of  argument,  of  course,  is  that  while  one  may  discover  many  physical  curves  that 
are  lines  of  curvature,  another  might  find  many  other  curves  that  are  not.  The  a  prion  justification  for 
assuming  that  physical  curves  arc  lines  of  curvature  docs  not  follow  by  such  argument.  In  die  synthetic  world 
that  argument  is  rather  compelling,  but  not  so  in  the  natural  world.  The  likelihood  of  being  correct  in 
assuming  a  given  image  contour  corresponds  to  a  line  of  curvature  is  significantly  improved,  howr  -r,  if  one 


explicitly  excludes  from  consideration  certain  classes  of  image  contours  (hat  arc  likely  not  to  lines  of 
curvature.  One  class  of  contours  to  disregard  arc  those  dial  exhibit  systematic  evidence  of  fore  inning, 

such  as  do  the  texture  contours  discussed  earlier.  That  is.  die  characteristic  evidence  of  for  toning 

associated  w  ith  the  projection  of  random  curves  across  approximately  planar  surfaces  can  be  usr  is  a 


"triggering  condition”  for  estimating  surface  orientation  from  foreshortening,  and  simultaneously,  as  a 
condition  for  excluding  them  from  consideration  as  lines  of  curvature. 

2.6.4  Discussion:  1'hc  line  of  reasoning  just  given  can  be  c.isily  summarized  as  follows.  Suppose  one  is 
presented  with  parallel  contours  in  an  image,  as  in  figure  1.  They  almost  certainly  correspond  to  parallel 
contours  in  3-1)  by  general  position  of  viewpoint.  The  chance  is  negligible  of  viewing  two  differently  shaped 
(thus  non-parallel)  contours  from  a  perspective  which  causes  their  projections  to  be  identical  and  to  differ 
only  by  a  translation  in  die  image  plane.  Moreover,  without  evidence  to  the  contrary,  one  can  assume  diat  die 
physical  placement  of  the  contours  does  not  conspire  with  the  underlying  surface  geometry  to  mislead  die 
viewer.  The  precise  placement  of  die  contours  on  the  surface  is  assumed  not  to  be  critical;  dicy  could  be 
displaced  and  appear  very  similar.  It  follows  that  die  surface  is  a  cylinder  in  the  vicinity  of  the  physical 
contours.  (Intuitively,  that  means  that  one  can  slide  one  curve  along  a  straight  line  on  ilw  surface,  without 
dips  or  rises,  until  it  superimposes  over  die  other.)  Parallel  image  contours  therefore  imply  cylindricality. 
Now.  since  the  contours  arc  also  assumed  to  correspond  to  lines  of  curvature  across  die  surface,  it  follows  that 
they  arc  planar  sections  and  geodesics.  The  parallel  curves  correspond  to  parallel  planar,  nonnal  sections  of 
die  cylinder. 

Note  that  we  shirt  with  only  three  assumptions:  i)  general  position  of  viewpoint,  ii)  general  position  of 
contour,  and  Hi)  die  line  of  curvature  assumption.  These  assumptions  logically  entail  die  geometric 
constraints  of  planarity,  geodesic,  perpendicular  intersection,  and  cylindricality.  Thus,  if  die  three 
assumptions  are  incorporated  in  human  vision,  the  geometric  constraints  follow  as  consequences  and  need  not 
be  independently  motivated.  Next  we  discuss  how  the  constraints  allow  surface  orientation  to  be  determined. 

2.7  Inferring  surface  orientation 

With  reference  to  figure  4.  consider  the  case  of  multiple  parallel  surface  contours.  We  assume 

general  position  of  viewpoint  (1) 

genera!  position  of  contour  (2) 
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Figure  4  Parallel  surface  coniours  {<;)  arc  interpreted  as  parallel  in  3-D  and  as  lying  on  a  cylindrical  surface.  Al  two  points  along  one 
contour  the  rulings  arc  shown  to  intersect  Note  that  the  rulings  are  parallel  straigh:  lines,  l'ach  intersection  ls  the  oblique  projection 
of  a  right  angle  on  the  surface,  and  this  fact  constrains  the  range  of  possible  orientations  for  the  normal,  as  shown  in  b  Note  that 
where  the  angle  of  intersection  is  laige  the  surface  orientation  is  strongly  constrained,  and  the  bisector  becomes  an  increasingly  good 
estimate  of  the  surface  tilt.  Provided  the  surface  orientation  is  solved  at  one  point  along  the  curve  it  can  be  solved  cverwhere.  see  text. 


T  is  a  line  of  (greatest)  curvature  (3) 

where !’  is  the  physical  curve  corresponding  to  a  given  contour  C  in  the  image.  We  will  represent  the  physical 
curve  T  parametrically  by  x  -  xr(s)  and  define 
tl  (s)  as  die  tangent  vector  of  T 
nf(s)  as  the  principal  normal  vector  of  T 
p(s)  as  the  normal  to  2  at  points  along  T. 

Consider  two  contours  C,  and  C.  corresponding  to  physical  curves  T,  and  T.  across  die  surface  2.  From  (1)  we 
have 

cue  =*  r  n r  (4) 

i  j  i  i 

and  (2)  and  (4)  yields 

2  cylindrical.  (5) 

That  is.  parallel  curves  imply  a  cylindrical  surface,  provided  general  position  of  both  viewpoint  and  contour 
placement.  From  (3)  and  (5)  it  also  follows  that 
T  is  geodesic 


(6) 
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T  is  planar,  lying  in  some  plane  n  (7) 

It  follows,  therefore,  that  the  surface  normal  v  lies  in  the  plane  H  and  is  perpendicular  to  I'.  As  the  next  step 
in  deriving  v  we  examine  die  straight  lines  on  die  cylinder,  the  rulings. 

The  ndings,  recall,  arc  lines  of  least  curvature  on  a  cylinder,  and  since  T  is  a  line  of  greatest  curvature,  all 
rulings  arc  perpendicular  to  T.  Moreover,  on  a  cylinder  all  rulings  arc  parallel,  as  noted  earlier.  1  lencc,  if  we 
refer  to  a  tilling  as  A,  we  have  diat 

A  is  straight  (8) 

a  ±  r  (9) 

A.  II  A.  V  i,j.  (10) 

Since  all  die  rulings  on  a  cylinder  are  straight  and  parallel,  they  share  die  same  tangent  vector  tA. 

The  tangent  1  to  the  ruling  A  is  perpendicular  to  TI.  the  plane  containing  I\  That  is 

tv±n  (ii) 

To  see  this,  observe  that  T  is  geodesic  (6)  and  planar  (7)  hence  the  plane  II  is  perpendicular  to  the  tangent 
plane  to  the  surface  2.  Since  the  tilling  A  lies  in  the  tangent  plane  and  is  perpendicular  to  T,  it  is  also 
perpendicular  to  H.  (Consequently  die  tangent  to  the  ruling  is  equivalently  the  binomial  to  T,  and  the 
tangent,  principal  normal,  and  binomial  to  P  arc  mutually  orthogonal  at  each  point  along  the  curve.) 

Thus  we  now  may  relate  die  surface  normal  i<(s)  along  the  curve  T  in  a  simple  manner  to  die  tangents  to 
the  curve  and  the  ruling 

ids)  =  nr(s)  =  tA(s)  X  tr(s).  (12) 

Thus  if  the  path  of  the  curve  T  were  determined  in  3-D,  the  surface  orientation  along  that  path  could  be 
computed  straightforwardly.  In  fact  determining  die  path  of  P  in  3-1)  reduces  to  die  problem  of  determining 
the  plane  II  containing  T  (since  T  is  simply  die  orthographic  projection  of  C  onto  II).  Provided  11  can  be 
determined,  die  surface  normal  is  computable  cidier  from  the  principal  normal  to  P,  or  from  the  cross 
product  relation  in  (12). 

One  possible  strategy  for  solving  for  the  surface  orientation,  therefore,  would  involve  inferring  the  spatial 
orientation  of  die  plane  containing  f\  We  shall  show  later,  however,  that  die  human  visual  system  probably 
does  not  use  diis  strategy.  One  dial  is  more  likely  used  is  described  by  the  following. 

Consider  the  intersection  of  die  line  of  greatest  curvature  T  and  the  line  of  least  curvature,  or  ruling.  A.  In 
the  image  T  projects  as  die  contour  C,  and  die  ruling  A  projects  as  the  straight  line  R.  Figure  4 a  shows  two 
rulings,  which  appear  as  parallel  straight  lines,  each  intersecting  the  contour  C  at  some  angle  / 3 .  The  rulings, 
while  not  visible  in  figure  1,  can  be  reconstructed  by  a  simple  geometric  construedon  that  follows  from  (8-10), 
i.c.  die  ndings  on  the  surface  arc  parallel  straight  lines  that  intersect  each  line  of  greatest  curvature  at  a 
constant  (right)  angle.  Consequently  the  projection  of  a  ruling  would  be  a  straight  line  that  intersects 
successive  image  contours  at  a  constant  angle  ft,  and  all  rulings  would  be  parallel  in  orthographic  projection. 
Thus  it  is  sufficient  to  identify  points  on  successive  contours  having  parallel  tangents,  and  to  connect  diose 
points  with  straight  lines  diat  are  dictnsclvcs  parallel,  flic  correspondence  between  parallel  lines  can  be 
determined  by  a  simple  local  computation;  see  (Stevens  1981a]  for  more  detail. 

We  impose  a  Cartesian  coordinate  frame  whose  x-y  plane  lies  parallel  to  the  image  and  whose  x-axis  is 
aligned  with  R.  Construct  tangent  vectors  to  V  and  A  scaled  such  that  they  project  as  unit  vectors  in  the 
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image  plane: 

tr(so)  =  {1.0>a} 
tA(s0)  =  {cos 0O,  Sin/3a,  b} 

where  0o  is  the  angle  of  intersection  at  the  given  point.  ’Hie  quantities  a  and  b  arc  the  unknown  components 
along  the  z-axis  (perpendicular  to  die  image  plane).  'ITic  surface  normal  v  is  their  cross  product: 

s0)  =  {-o  sin0o,  acos0o  -  b ,  sin0o}  (13) 

and  the  tilt  r  and  slant  a  arc: 


t  =  tan 


a  =  cos 1 


( v  +  V  +  'tv 


where  v  ,  v  ,  and  v  arc  die  tlircc  components  of  the  normal  vector, 
x  y  i 

The  expression  for  the  normal  in  (13)  carries  two  unknowns.  This  reflects  the  two  degrees  of  freedom  of 
sui  face  orientation  when  no  restrictions  arc  imposed.  Now,  given  (9), 

W  •  ‘  =  0 

from  which  we  have 

cosfl 

b  - - & 

a 

Substituting  b  into  (13)  gives 

Wso)  =  {-fl  sin/1,,,  cos/)0,  sin0o)  .  (15) 

and  therefore  the  surface  orientation  is  determined  up  to  only  one  unknown,  a.  Thus  perpendicularity 
between  the  lines  of  greatest  and  least  curvature  removes  one  degree  of  freedom  of  surface  orientation. 

The  magnitude  of  this  constraint  depends  on  the  angle  of  intersection  0.  For  any  angle  of  intersection  the 
tilt  is  constrained  to  lie  within  the  perpendiculars  to  R  and  C,  and  the  slant  is  constrained  accordingly.  As  0 
approaches  180°  die  tilt  approaches  the  bisector  of  the  angle  and  the  slant  approaches  90°.  The  restriction  on 
die  range  of  possible  tilts  becomes  appreciable  as  0  becomes  large: 

0  =  135°:  a  =  77.75±  12.25°  t  =  the  bisector  ±22.5°. 

0  =  150°:  a  =  82.25±7.758  r  =  the  bisector  ±15.0° 

0  =  170°:  a  =  87.49±2.51°  r  =  die  bisector  ±5.0°. 

Hence  the  bisector  of  the  angle  of  intersection  becomes  an  increasingly  good  estimate  of  the  surface  dlt. 
(Incidentally,  for  any  obtuse  0  die  bisector  choice  for  die  tilt  corresponds  to  the  surface  orientation  with  the 
least  slant.) 

Returning  to  figure  4o,  observe  that  the  angle  of  intersection  varies  along  the  curve;  0  is  greatest  at  the 
intersection  shown  on  the  right  (call  that  point  0)  and  least  at  die  intersection  on  die  left  (point  1).  Thus  at 
point  0  the  surface  orientation  is  most  strongly  constrained  by  the  perpendicularity.  Specifically  for  figure  4, 
0  =  167.1°  at  point  0,  and  the  tilt  is  128.5±6.5°.  Ihc  bisector  of  die  angle  of  intersection  therefore  is  a  good 
estimate  of  die  local  surface  dlt  at  point  0. 

At  point  1.  on  die  other  hand,  0  =  102.9°  and  the  tilt  is  only  constrained  to  96.4±38.6°.  Nonetheless, 
having  determined  the  orientation  at  point  0  will  allow  the  tilt  to  be  solved  here  and  at  all  other  points, 
t Intuitively,  since  the  Y  is  both  geodesic  and  planar,  the  surface  normal  is  constrained  to  rotate  in  the 
osculating  plane,  and  once  it  is  known  at  one  point  on  the  curve,  it  rotates  in  a  predictable  manner  along  the 
curve.) 
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Point  1  is  defined  as  x  =  \(s ).  and  the  tangents  to  the  curve  and  die  ruling  are 
t,.(s,)  -  {1,0, 
tA(s,)  =  {cos/?t,  sin /?,,<■} 

where  (3^  is  the  angle  of  intersection  at  that  point.  Observe  that  since  the  rulings  arc  parallel,  tr(s  )  =  tr(so). 
The  surface  normal  is,  again,  the  cross  product  of  die  two  tangent  vectors: 

^(Sj)  =  {-as\n/ir  acos/lj  -c,  sin/Jj}  (16) 

where 

cos  fl, 

c  = - “  (17) 

a 

Note  diat  the  one  remaining  variable  at  this  point,  c,  is  related  to  a.  But  observe  also  diat  a  is  simply  related  to 
r: 

T  =  tan  1  ^  =  tan  1  (-cot/?,  iJ~ ^). 

Thus  a  can  be  solved  in  terms  of  the  known  till  r  and  die  obtuse  angle  /?, 

3  =  "8» 
Finally,  substituting  a  into  (17)  and  (16)  completely  determines  the  surface  orientation. 

Figure  4c  shows  die  bisector  estimate  for  the  tilt  at  point  0  (t  =  128.5°,  a  =  83.5°),  and  die  surface 
orientation  solved  for  point  1  (r  =  6*).8°,  a  =  47.4°).  This  was  determined  simply  by  die  computation  just 
described.  As  a  first  demonstration  of  the  implementation,  figure  4J  shows  how  circles  would  appear  if  lying 
on  the  surface  along  the  padi  of  die  curve,  demonstrating  diat  the  orientation  can  be  determined  for  all  points 
along  the  curve. 

To  review,  the  surface  orientation  is  strongly  constrained  in  places  where  F  intersects  a  ruling  at  a  very 
large  obtuse  angle.  Knowing  die  orientation  at  diat  point  means  that  the  orientation  is  known  everywhere 
along  die  curve.  This  might  be  described  as  extrapolation  of  the  solution  from  places  where  it  is  strongly 
constrained.  But  note  that  die  solution  at  any  point  is  locally  determined  given  one  global  parameter  (which 
wc  defined  as  die  variable  a). 

Before  examining  further  demonstrations  of  diis  niediod,  it  is  instructive  to  examine,  rather  informally, 
whether  our  3-D  interpretations  seem  consistent  with  die  underlying  geometric  constraints,  namely,  that  die 
contour  is  planar,  geodesic  and  a  line  of  curvature,  and  that  die  surface  is  cylindrical. 


3.  Qualitative  observations 


3.1  Smooth  curves  generally  appear  planar  in  3-1) 

A  solitary  curve  in  monocular  presentation  usually  appears  planar  and  in  3-D.  The  curve  in  figure  5a,  for 
example,  seems  to  lie  in  a  different  plane  than  the  plane  of  the  page.  (It  is  suggested  that  this  and  subsequent 
figures  be  viewed  moriocularly  with  sufficient  viewing  time  allowed  for  depth  impressions  to  develop.)  In  fact 
it  is  quite  difficult  to  perceive  the  torsion  of  a  solitary  twisting  curve  from  a  single  orthographic  projection  - 
figure  5b  is  the  projection  of  a  helix,  although  it  is  difficult  to  see  it  as  such.  The  intcipretation  as  a  twisting 
space  curve  is  easier  to  achieve  if  the  curve  projects  as  a  self-intersecting  image  curve.  The  helix  in  figure  5c, 
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Figure  $.  A  smooth  curve  is  usually  seen  as  planar  (a).  (Observe  this  and  subsequent  figures  monocularly.  and  allow  sufficient 
viewing  time  for  depth  impressions  to  develop.)  ft  is  quite  difficult  to  perceive  the  torsion  of  a  solitary  twisting  curve  from  a  single 
image  unless,  say.  it  is  self-intersecting.  ( 'oinpaic  b  and  c.  both  of  which  are  are  helices. 


for  example,  is  more  obviously  twisting  in  space.4 

3.2  Intersections  are  generally  seen  as  right  angle  intersections  in  3-D 

A  simple  oblique  intersection  appears  to  be  the  image  of  a  right-angle  intersection  in  space  (figure  6).  Hie 
literal  interpretation  of  either  intersection  in  figure  6  as  an  oblique  (non-right  angle)  intersection  in  the  plane 


f  igure  6.  Fxamplc  of  simple  intersections  that  arc  strongly  interpreted  as  right-angle  intersections  in  3-0. 


4  Incidentally,  plananty  seems  to  be  applied  only  piecewise  along  extended  contours  An  image  curve  without  inflection  points  in 
ctitx  mire  or  discontinuities  in  tangent  is  usually  seen  as  a  single  planar  curve  Rut  at  such  points  the  curve  might  appear  to  twist 
ahiupth  As  a  result  ilte  overall  curve  appears  lo  he  composed  of  piecewise- planar  arcs  in  3  I >  In  large  pail  the  spatial  oiieulation  of 
each  individual  arc  seems  to  be  independently  determined  Ihc  leader  h  invited  to  draw  various  isolated  curves  to  further  examine  the 
tendency  for  plananty  and  the  instances  where  curves  are  seen  as  piecew  ise  planar  or  to  have  torsion. 
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of  the  page  is  very  difficult  to  achieve  in  monocular  or  even  binocular  presentation.  Ibis  tendency  to  take  a 
right-angle  interpretation  has  been  observed  in  various  contexts,  including  illusions  (see  r.g.  [I.uckicsh  1965; 
Robinson  1972J),  and  in  conjuction  with  3-1)  interpretations  of  geometric  figures  (Attncavc  &  Frost  1969; 
Attncave  1972;  Perkins  1972;  Finch  1975;  Shepard  1979;  Stevens  1982].  Further  instances  of  perceived 
right-angles  occur  in  other  figures  in  this  article. 

3.3  intersections  generate  apparent  surfaces  in  3-1) 

A  simple  curve  having  only  a  weak  3-D  interpretation  (figure  7 a)  suddenly  provides  a  strong  impression  of  a 
surface  when  intersected  by  a  line  segment.  The  apparent  surface  shape  is  substantially  independent  of 
whether  the  line  intersects  an  endpoint  (figure  lb.  c.  f,  and  g)  or  some  interior  point  (figure  7c/ and  e).  Note 
further,  by  comparing  figures  lb.  c/and  / with  figures  7r.  c  and  g.  that  the  curve  can  be  interpreted  either  as  a 
physical  boundary  or  an  interior  marking  without  substantially  affecting  the  apparent  surface  shape.  In  either 
case  the  surface  appears  to  be  cylindrical.  (In  figure  7  all  intersecting  lines  have  the  same  orientation;  die 
intersection  angle  critically  affects  the  apparent  surface  --  see  later.) 

3.4  Parallel  curves  appear  to  lie  on  cylindrical  surfaces 

Two  arbitrary  smooth  curves  that  arc  parallel  (i.e.  differ  only  by  a  translation)  arc  sufficient  to  elicit  a  3-1) 
interpretation.  The  curves  are  seen  as  llie  two  edges  of  a  ribbon-Iikc  surface  (figure  8o)  Note  that  additional 
parallel  curves  enhances  the  impression  of  a  surface  but  the  apparent  surface  shape  is  substantially  unchanged 
(figure  8 b).  Note  further  that  the  spacing  between  curves  is  immaterial.  In  figure  9  die  curves  arc  precisely 
parallel  but  die  spacing  between  curves  is  random.  Under  careful  inspection  figure  9  defines  die  same 
cylindrical  surface  as  seen  in  figure  8 b. 

The  random  spacing  between  die  parallel  contours,  however,  produces  an  illusion  of  shading,  which  in 
turn  may  purturb  the  impression  of  viewing  a  cylindrical  surface.  First  consider  figure  8 b.  where  die  line 
density  varies  across  the  pattern.  There  is  a  diagonal  swath  across  the  figure  where  die  curves  nearly  touch, 
w'hich  appears  shaded  in  parafoveal  vision.  The  shading  suggests  surface  curvature  along  that  diagonal,  which 
likely  enhances  the  impression  of  surface  shape.  (Shading  due  to  variations  in  line  density  probably 
contributes  to  the  impression  of  three-dimensionality  in  these  patterns,  but  as  figure  2  demonstrates,  shading 
is  not  wholly  responsible.)  Returning  to  figure  9,  besides  die  shading  along  die  diagonal,  the  random  spacing 
of  die  parallel  lines  produces  additional  shading  that  suggests  slight  ripples  running  perpendicular  to  the 
major  troughs  and  crests  of  the  surface.  When  the  surface  shape  is  examined  closely,  however,  these  shading 
effects  largely  vanish. 

Finally,  consider  figure  10.  in  which  straight  line  segments  intersect  a  given  curve.  If  the  lines  segments  are 
parallel  the  surface  appears  cylindrical  (figure  10c/).  If  they  diverge  slightly  (figure  10 b)  we  attribute  the 
divergence  to  perspective.  Thus  the  same  surface  is  seen  in  each  case,  one  in  ordiographic  projection,  die 
other  in  perspective  (i.e.  from  close  by).  It  is  much  more  difficult  to  take  the  alternative  interpretation  dial  the 
surface  is  developable  but  twisted,  as  if  a  sheet  of  paper  were  twisted  into  a  helicoid.  (Incidentally,  observe 
that  die  parallel  rulings  in  figure  10a  might  actually  appear  to  converge  in  3-D.) 
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Figure  7.  In  a  ihe  simple  curve  has  only  a  weak  3-1)  interpretation.  But  the  addition  of  lines  that  intersect  it,  in  fr-g.  cause  us  to  sec  a 
surface  that  resembles  a  gently  curved  sheet  of  paper.  To  demonstrate  that  the  curve  can  be  interpreted  as  cither  a  physical  boundary 
or  an  interior  marking,  without  changing  the  apparent  surface  shape,  compare  b-c,  d-c  and  / g.  To  demonstrate  that  the  location  of  the 
intersection  along  the  arc  is  not  critical,  compare  b-d-f  and  compare  c-e-g. 


3.5 1’nrallcl  runes  appear  to  be  lines  of  greatest  curvature  on  cylindrical  surfaces 

Referring  back  to  all  preceding  figures,  the  general  impression  is  of  curves  that  follow  the  greatest  curvature 
across  the  surface.  This  is  supported  particularly  in  figures  4.  7,  and  10,  where  the  straight  lines  (that  follow 
the  direction  of  least  curvature  across  die  surface)  appear  to  be  perpendicular  to  the  curves.  Also  supporting 
this  is  the  following:  the  curves  appear  to  be  planar  normal  sections  of  cylinders  and  all  planar  normal 
sections  of  a  cylinder  arc  lines  of  curvature. 

Moreover,  consider  the  cylindrical  surfaces  depicted  in  figure  11.  The  curves  appear  to  follow  the  direction 
of  greatest  curvature  on  the  surface.  Informally,  they  resemble  flow  lines  over  a  waterfall,  or  sections  of  an 
airfoil.  In  each  case  they  appear  as  planar  normal  sections,  and  as  just  mentioned,  it  therefore  follows  that 
they  arc  lines  of  greatest  curvature  on  the  cylinder. 

figure  1 1  also  demonstrates  an  important  point  regarding  how  the  curves  themselves  are  seen  in  3-D. 
Each  curve  in  figure  11  is  an  air  of  an  ellipse.  Usually,  an  isolated  arc  of  an  ellipse  assumes  the  3-D 
orientation  corresponding  to  an  arc  of  a  slanted  circle,  just  as  a  complete  ellipse  is  seen  as  a  slanted  circle  in 
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Figure  8.  Two  smooth  curves  that  arc  parallel  suggest  a  ribbon-like,  cylindrical  surface  (a).  The  impression  of  a  surface  is  enhanced 
but  the  apparent  surface  shape  is  unchanged  when  additional  parallel  curves  are  added  ( b ). 


3-D.J  (Hence  an  ellipse  or  a  section  of  an  ellipse  appears  as  a  section  of  a  constant-curvature  curve  in  3-0.) 
Rut  when  the  ellipse  sections  arc  arrayed  in  parallel  fashion,  as  in  figure  11,  the  dominant  impression  is  of 


5.  To  some  extent  this  an  be  observed  by  scrutinizing  the  bottom-most  ellipse  of  each  surface  in  figure  11  mil  note  that  the  other 
ellipses  assume  a  different  orientation  in  3-1). 
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b 


figure  10.  In  a.  where  the  intersecting  lines  are  parallel,  the  surface  appears  cylindrical.  In  b.  where  they  are  not,  the  same  surface  is 
seen,  but  in  perspective. 
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cur\es  lying  on  a  surface  such  that  the  normal  curvature  is  not  constant  along  the  curve.  The  spatial 
orientation  of  die  osculating  plane  of  each  curve  is  apparently  not  dominated  by  the  shape  of  the  individual 
projected  curve.  (1'his  suggests  dial  the  human  visual  system  does  not  first  determine  the  orientation  of  the 
plane  containing  the  curve,  then  use  that  to  determine  the  surface  orientation,  as  mentioned  earlier.)  Ihe 
bottom  left  surface  in  figure  1 1  presents  a  particularly  clear  demonstration  since  the  drawn  curves  are,  in  fact, 
arcs  of  circles.  Rather  titan  appear  to  lie  in  planes  parallel  to  the  printed  page,  they  appear  to  be  cross 
sections  seen  from  an  oblique  viewpoint. 

Figure  12  demonstrates  that  parallel  copies  of  virtually  any  curve  can  induce  the  impression  of  a  cylindrical 
surface.  Here,  rather  than  precise  sinusoids  or  arcs  of  ellipses,  die  repeating  curve  is  a  spline  that  was  drawn 
through  some  points  that  were  chosen  arbitrarily.  These  parallel  lines  appear  to  lie  on  a  surface  that  is 
precisely  cylindrical  (a  straight  line  placed  on  die  surface  parallel  to  the  troughs  and  ridges  would  touch  the 
sm  face  at  all  points  along  its  length).6 

Finally,  observe  in  figure  13  that  ihe  global  surface  shape  is  not  restricted  to  being  strictly  cylindrical.  1'his 
surface  appears  doubly  curved,  and  resembles  concentric  ripples  on  a  pool.  Hie  curves  arc  locally,  but  not 
globally,  parallel.  It  is  conjectured  that  doubly-curved  surfaces  such  as  that  depicted  by  figure  13  arc 
perceived  by  the  same  processes  dial  derive  die  3-1)  shape  of  cylindrical  surfaces  from  parallel  contours.  ITiat 
is,  die  method  involves  a  local  correspondence  between  parallel  segments  of  contours  (to  estimate  the  angle  /), 


Figure  12.  The  irregular  cylindrical  surface  is  suggested  by  multiple  parallel  copies  of  a  smooth  spline  drawn  through 
artmrarily-choscn  points.  The  curves  arc  interpreted  as  norma!  sections  of  the  cylinder,  and  the  ridges  and  troughs  run  perpendicular 
to  the  curves,  thereby  following  the  lines  of  least  curvature  or  rulings  of  the  surface. 


6  Pxcept  along  some  ridges  where  the  surface  may  appears  to  dip  slightly  between  the  contours  This  effect,  curiously,  scents  associated 
only  with  apparent  ridges  therefore  when  the  the  figure  reverses  in  depth  so  that  what  were  seen  as  (convex)  ridges  now  appear  as 
(concave)  troughs,  the  surface  along  die  trough  appears  precisely  cylindrical  The  depth  reversal  can  be  induced  most  easily  by  simply 
inverting  the  figure  (since  we  tend  to  intciprel  distance  as  increasing  as  one  scans  from  bottom  to  top  on  Ihe  figure,  diis  biases  the  depth 
interpretation  in  these  figures).  Where  the  apparent  surface  undulates  as  one  scans  along  a  ridge,  the  contours  seem  to  be  interpreted  as 
bounding  (or  silhouette)  contours  [Marr  197')  of  slight  peaks  Ihe  impression  is  enhanced  by  obscuring  all  but  a  single  ndge  and  its 
flanking  slopes  on  cither  side:  each  individual  curve  niav  appear  to  be  die  outline  of  a  small  conical  hill  Again,  when  the  figure  is 
inverted,  this  -cup  looks  like  a  culvert  Ur.  cvlindiical);  n  is  i|iuie  difficult  to  interpret  it  as  an  inverted  view  ol  a  tow  of  small  conical  hills 
Ibis  curious  phenomenon  seems  independent  of  the  tendency  to  interpret  parallel  contours  as  lying  on  a  cylindrical  surface 
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Figure  13  The  doubly-curved  surface  resembling  ripples  on  a  pool  arise  from  sinusoids  having  a  gradually- varying  phase  shift,  lhc 
curves  arc  locally,  but  not  globally,  parallel.  Arbitrary  doubly -curved  surfaces  arc  amenable  to  a  local  approximation  by  cylinders 
provided  the  contours  are  sufficiently  closely  spaced. 


etc.,  as  discussed  in  section  2.7)  and  this  method  is  tolerant  of  gradual  variations  in  the  correspondence  across 
the  surface.  Thus,  provided  the  contours  are  sufficiently  closely  spaced,  arbitrary  surfaces  can  be  analyzed  by 
this  method. 

4.  Demonstration  of  the  implementation 

Informal  observation  suggests  that  we  interpret  parallel  curves  as  planar,  geodesic  and  as  lines  of  curvature 
across  cylindrical  surfaces.  These  observations  arc  consistent  with  the  hypothesis  that  the  human  visual 
system  incorporates  assumptions  (1-3).  Stronger  support  is  provided  by  a  computer  implementation  of  the 
method  defined  by  (14-18).  The  surface  orientation  predicted  by  this  method  is  in  dose  agreement  with  the 
apparent  surface  orientation. 

'The  implementation  computes  the  surface  normal  at  points  along  a  curve  from  image  configurations  such 
as  we  have  examined,  lhc  bisector  of  die  angle  is  used  as  an  estimate  of  the  surface  tilt  r  at  the  point  where 
fi  is  largest.  The  tilt  estimate  allows  the  variable  a  to  be  computed  from  (18).  Given  a.  die  normal  at  diat 
point  is  completely  specified,  so  die  slant  can  also  be  determined  from  (14).  (Note  diat  the  bisector  solution  is 
but  one  of  several  independent  means  for  csdmating  the  orientadon  at  a  point,  from  which  a  may  be 
determined.)  For  any  odicr  point  along  the  contour,  the  value  of  c  may  be  computed  from  (17)  given  a  and 
the  measured  value  of  iat  that  point.  Given  cat  that  point,  die  surface  normal  (16)  is  also  determined,  so  that 
the  lilt  and  slant  at  that  point  can  also  be  computed  from  (14).  lhc  computation  of  surface  orientation  at  any 
point  is  therefore  a  simple  function  of  die  local  parameter  /?  and  the  global  parameter  a. 

The  surface  orientation  computed  by  this  mediod  can  be  graphically  depicted  by  superimposing  a  directed 
line  segment  diat  represents  die  local  surface  normal  (figure  4c)  or,  say,  by  computing  how  a  small  circle  lying 
on  the  surface  would  project  onto  the  image  plane,  (figure  4 it).  The  local  surface  orientation  predicted  by  this 
mediod  would  seem  in  close  correspondence  widi  diat  perceived  by  the  human  observer  if  die  superimposed 
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line  segments  appear  perpendicular  to  the  surface  or  the  superimposed  ellipses  appear  to  be  circles  that  lie  in 
the  tangent  plane  of  the  surface.  A  demonstration  of  both  schemes  is  given  in  figure  14.  Figure  14 a  shows  a 
curve  intersected  by  parallel  straight  line  segments.  The  implementation  computed  die  surface  normals 
shown  in  figure  146,  and  the  equivalent  foreshortened  circles  in  figure  14c.  One  may  compare  the  apparent 
surface  orientation  in  figure  14c/  with  the  predictions  below  it.  (The 'figure  should  be  viewed  inonocularly, 
oriented  perpendicular  to  the  line  of  sight,  and  studied  for  a  few  seconds  in  order  to  develop  a  strong 
impression  of  three-dimensionality.) 

While  the  depiction  of  a  surface  norma!  by  a  short  line  segment  is  an  excellent  means  for  indicating  local 
surface  tilt,  the  slant,  indicated  by  die  length  of  die  line  segment,  is  much  harder  to  judge  from  die  displayed 
line.  While  we  are  sensitive  to  the  length  of  the  depicted  surface  normals  (because  they  appear  obviously 
incorrect  when  all  arc  displayed  as  with  equal  length  rather  than  foreshortened  by  die  sine  of  die  slant  angle) 
it  is  difficult  to  make  subtle  judgments  of  slant  from  variations  in  die  projected  line  lengths.7  Furthermore, 
the  line  segments  diat  represent  surface  normals  may  influence  the  apparent  surface  shape.  In  figure  14c,  the 
addition  of  short  line  segments  to  die  simple  configuration  of  figure  14c/  brings  a  complicating  factor.  As 
noted  earlier,  we  have  a  tendency  to  interpret  intersecting  straight  lines  segments  as  perpendicular  and 
equal-length  in  3-1).  Hence  the  added  line  segments  in  figure  14c  have  a  strong  potential  for  influencing  the 
spatial  interpretation  of  die  basic  figure.  ( ITiere  seems  to  be  less  interaction  when  die  depicted  surface  normal 


Figure  14  The  configurations  in  a  appear  in  3-D.  Compare  the  spatial  orientation  at  each  intersection  point  with  that  predicted  by  the 
inipIcmcntaUon.  Orientation  is  depicted  in  6  by  the  projection  of  a  unit  surface  normal,  and  in  c  by  the  projection  of  a  circle  lying 
(lush  on  the  surface. 


7  When  one  considers  what  inferring  local  slant  tn  this  means  would  entail,  this  obscnaiion  is  not  surprising  Note  how  little  the  line 
lengths  vary  in  figure  14c  compared  lo  the  differences  in  foreshortening  to  the  corresponding  ellipses  in  figure  146. 
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1  igure  15  The  shape  and  orientation  of  the  ellipses  in  a  arc  predicted  by  the  theory  and  appear  as  circular  disks,  each  lying  in  the 

O 

tangent  plane  to  the  surface  at  that  point  In  b  the  ellipses  arc  systematically  rotated  10  counterclockwise  relative  to  the 

O  O 

Corresponding  disks  in  a.  Similarly,  in  c  and  d  the  ellipses  arc  rotated  10  and  20  clockwise.  Observe  that  when  the  orientation  of 
I  lie  ellipse  is  inappropriate  the  interpretation  is  either  of  a  circular  disk  touching  the  surface  at  only  one  point  on  its  perimeter,  or  of 
an  ellipse  drawn  on  the  surface. 


is  presented  only  briefly  while  continuously  viewing  the  stimulus  figure  [Stevens  1982].) 

Consider  now  the  use  of  ellipses  to  depict  the  surface  orientation  predicted  by  the  implementation.  An 
ellipse  of  given  shape  and  orientation  is  very  effective  in  suggesting  a  circular  disk  of  die  corresponding 
surface  orientation.  Superimposed  on  the  image  of  a  surface,  an  ellipse  is  inteipretcd  as  a  disk  that  contacts 
the  apparent  surface,  and  if  oriented  appropriately,  the  disk  appears  to  lie  in  the  tangent  plane  to  the  surface 
at  that  point.  The  ellipse  may  be  used  as  a  probe  of  apparent  surface  orientation,  since  ellipses  of 
inappropriate  orientation  or  shape  will  not  appear  to  lie  on  the  surface.  To  demonstrate,  figure  15a  shows  a 
row  of  ellipses,  each  of  which  appears,  to  most  observers,  to  lie  flush  on  the  surface.  In  figure  15 b  the  ellipses 
arc  systematically  rotated  10°  counterclockwise  relative  to  the  corresponding  ellipses  in  figure  15a.  Similarly, 
in  figures  15c  and  lSJthe  ellipses  are  rotated  10°  and  20°  clockwise.  Observe  that  when  the  orientation  of  the 
ellipse  is  inappropriate  the  interpretation  is  either  of  a  circular  disk  touching  the  surface  at  only  one  point  on 
its  perimeter,  or  of  an  ellipse  drawn  on  the  surface.  In  figure  15  the  apparent  tilt  of  the  disks  was  varied, 
holding  slant  constant.  Figure  16  varies  the  slant  while  holding  tilt  constant.  Figure  16a  (which  is  identical  to 
figure  15a)  shows  ellipses  that  appear  to  be  circles  on  the  surface,  while  the  ellipses  in  figure  166  correspond 
to  disks  that  arc  systematically  slanted  and  additional  10°.  and  in  figures  16c  and  16 d  the  disks  at  lamed  by 
10°  and  20°  less  than  the  corresponding  disks  in  16a.  Note  that  when  the  disks  arc  inappu.priaU.  •  slanted 
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I  igure  16.  In  a  (which  is  identical  to  figure  15a)  the 'ellipses  appear  to  be  circles  on  the  surface.  The  ellipses  in  b  correspond  to  disks 

e  o  o 

that  arc  systematically  slanted  and  additional  10  .  and  in  c  and  d  the  disks  arc  slanted  by  10  and  20  less  than  the  corresponding 
disks  in  a  Note  the  apparent  step-like  discontinuity  in  the  surface  in  c  and  J (in  a.  on  the  other  hand,  the  surface  appears  smooth  and 
continuous  across  the  row  of  disks) 


relative  to  the  surface  they  tend  to  introduce  an  apparent  step-like  discontinuity  in  die  surface  (see  figures  16c 
and  16c/).  In  figure  16cr.  on  the  oilier  hand,  die  surface  appears  smooth  and  continuous  across  the  row  of 
disks.  As  diese  figures  demonstrate,  we  arc  sensitive  to  "errors”  in  die  orientation  and  shape  of  the 
superimposed  ellipses,  which  correspond  to  discrepancies  of  (somewhat  less  than)  10°  in  tilt  and  slant  relative 
to  die  apparent  orientation  of  the  underlying  surface. 

The  local  surface  orientation  predicted  by  the  Uicory  is  demonstrated,  using  ellipses,  in  figures  17,  18.  and 
19.  Observe  that  in  each  instance  the  ellipses  appear  as  disks  lying  on  the  surface,  even  when  the  figure 
reverses  in  depdi.  Figure  17  shows  variations  in  apparent  surface  shape  with  changing  the  amplitude  of  the 
sinusoids.  Likewise,  in  figure  18  the  phase  between  adjacent  sinusoids  is  varied.  Finally,  figure  19  shows  the 
implementation  applied  to  die  cylinders  of  figure  11,  with  similar  results. 

§.  Discussion 

The  implementation  supports  die  hypothesis  that  our  visual  interpretation  is  constrained  by  treating  the 
image  cuncs  as  projections  of  lines  of  curvature  assuming  both  the  viewpoint  and  contour  placement  arc 
representative.  Moreover,  die  specific  mcdiod  of  estimating  surface  tilt  at  die  point  of  greatest  constraint  by 
the  bisector,  while  not  critical  to  the  overall  dieory.  seems  well  supported  by  the  implementation.  The 


a 
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Figure  17.  Sinusoid  paitems  of  differing  amplitude.  In  each  case  one  sinusoid  is  replaced  with  a  scries  of  ellipses  that  appear  to  lie 
Hush  on  the  subjective  surface.  Note  that  they  conunue  to  appear  appropriate  when  the  figure  suddenly  reverses  in  depth  or  as  one 
rotates  the  figure. 


implementation  was  intended  only  to  examine  these  two  points;  it  was  not  otherwise  meant  as  a  model  of  the 
corresponding  visual  process  in  human  vision. 

A  common  geometry  is  shared  by  die  configurations  consisting  of  parallel  curves  (figure  1),  a  single  curve 
intersected  by  parallel  straight  lines  (figure  10o)  and  the  rather  "minimal"  case  of  a  single  curve  intersected  by 
a  single  straight  (figure  7).  All  visually  suggest  cylindrical  surfaces,  where  the  curves  correspond  to  lines  of 
greatest  curvature  and  the  straight  lines  correspond  to  rulings.  We  saw  a  geometric  argument  for  why  parallel 
curves  imply  that  die  surface  is  cylindrical  in  section  2.7.  'ITic  same  argument  can  be  adapted  to  the  case  of  a 
single  curve  intersected  by  parallel  straight  lines  -  dial  geometry  also  implies  cylindricality.  The  only  curious 
ease  is  why  a  single  curve  intersected  by  a  straight  line  also  looks  like  a  cylinder.  At  most,  if  die  line  segment 
corresponds  to  a  ruling,  one  could  conclude  diat  the  surface  is  developable  but  not  necessarily  cylindrical. 
(Since  there  is  only  one  ruling  visible,  one  can  determine  from  this  configuration  whether  or  not  die  surface 
normal  twists  along  the  curve.) 

l  et  C  be  the  image  of  a  given  line  of  curvature  P  and  R  that  of  a  given  ruling  A.  Since  the  surface  is  a 
cylinder,  all  rulings  arc  mere  translations  of  one  another,  and  all  lines  of  curvature  arc  likewise  translations  of 
one  anodicr.  Thus  one  can  tcconstruct  how  all  rulings  and  lines  of  curvature  would  appear  in  the  image  by 
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a  b 


Figure  18.  Sinusoid  pallcrns  of  differing  phase  shift,  constant  amplitude  Note  that  the  ellipses  appear  appropriate  in  each  case,  even 
when  depth  reverses  and  as  the  figure  is  rotated. 


translating  C  in  either  direction  along  R,  and  by  translating  R  in  either  direction  along  C.  This  would 
reconstruct  die  orthographic  projection  of  an  orthogonal  net  or  grid  across  die  surface.  Thus  although  one 
starts  with  a  simple  configuration  of  a  few  parallel  curves  or  a  curve  intersected  by  one  or  two  straight  lines, 
one  can  effectively  reconstruct  how  a  dense  grid  across  die  surface  would  appear.  By  diis  means  die  surface 
orientation  can  be  determined  for  arbitrary  points  along  a  cylinder.  The  interpolation  of  surface  orientation  is 
conceivably  applied  to  doubly-curved  surfaces  (such  as  figure  13)  provided  they  may  be  approximated  locally 
by  cylinders.  It  remains  to  be  determined  how  well  a  simple  grid  interpolation  accounts  for  the  apparent 
surface  orientation  in  those  cases. 

To  summarize,  three  assumptions  (line  of  curvature  and  general  position  of  viewpoint  and  contour)  allow 
one  to  conclude  that  the  underlying  surface  is  cylindrical  when  presented  with  parallel  contours  and  certain 
other  configurations.  It  is  possible  to  reconstruct  the  projection  of  an  orthogonal  net  across  the  surface,  and 
provided  diat  the  surface  orientation  is  determined  at  one  point  on  the  surface  it  can  be  solved  everywhere 
else.  (  The  bisector  is  one  of  several  methods  that  may  be  used  to  determine  orientation  at  certain  points  in 
order  to  seed  the  computation.)  Thus  the  computation  of  surface  orientation  from  the  image  configurations 
we  have  examined  is,  at  least  in  principle,  amenable  to  a  very  simple  local  algorithm  requiring  but  one  global 
parameter. 
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